Automatic Differentiation for Riemannian Optimization on Low-Rank Matrix and Tensor-Train Manifolds

In scientific computing and machine learning applications, matrices more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since tensors fixed rank form smooth Riemannian manifolds, one popular tools for finding approximations is to use optimization. Nevertheless, efficient implementation gradients Hessians, required in optimization algorithms, a nontrivial task practice. Moreover, some cases, analytic formulas are not even available. this paper, we build upon automatic differentiation propose method that, given an function minimized, efficiently computes matrix-by-vector products between approximate Hessian vector.